METHOD FOR RESTORING COMPRESSED IMAGE OF IMAGE PROCESSING 
SYSTEM AND APPARATUS THEREFOR 

BACKGROUND OF THE INVENTION 

5 

1. Field of the Invention 

The present invention relates to an image process 
technique, and in particular to a . method for restoring a 

3 compressed image by using a hybrid motion compensation 

3 

J 10 discrete cosine transform (hybrid MC/DCT) mechanism, and an 

j 

3 apparatus therefor. 

J 

4 

2. Description of the Background Art 

3 

1 In general, image compression techniques, such as MPEG1 

a 

5 15 and MPEG2 employ a hybrid motion compensation discrete cosine 
J transform (hereinafter, referred to as "hybrid MC/DCT") 

mechanism in order to improve compression efficiency. The 
hybrid MC/DCT mechanism is roughly divided into an encoding 
process and a decoding process. In the encoding process, an 
20 original image is divided into a plurality of blocks in order 
to compress information in a spatial section, a second- 
dimensional discrete cosine transform is performed on each 
block, and redundancy information in the image or between the 
images is reduced by using the correlation on a time axis 
25 among the images in order to decrease information in a 

1 
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temporal section. In the decoding process, the steps of the 
encoding process are performed in a reverse order. An encoder 
and a decoder are necessary to carry out the hybrid MC/DCT 
mechanism. 

5 Figure 1 is a block diagram illustrating an image encoder 

according to a related art. As shown therein, an input image 
signal is subtracted from an image signal moved from and 
compensated by an image memory 9, passed through a first 
3 switching unit 2, and inputted to a DCT unit 3. The DCT unit 

J 10 3 performs a discrete cosine transform on the inputted image 

j 

3 signal. A quantization unit 4 quantizes the image signal, and 

i 

XT 

« outputs a DCT coefficient (q) . An inverse quantization unit 6 

J 

inversely quantizes the DCT coefficient (q) , and an inverse 
1 DCT unit 7 carries out an inverse discrete cosine transform 

3 

« 15 thereon, thereby restoring the original image signal. The 
3 restored image signal is added to an image signal restored in 

a previous stage by an adder 8, and inputted to an image 
memory 9. A controller 5 controls switching of the first and 
second switching units 2, 10, and transmits INTRA/ INTER 
20 information (p=mtype; flag for INTRA/ INTER) , transmission 
information (t; flag for transmitted or not) , and quantization 
information (qz=Qp; quantizer indication) to a decoder (not 
shown in Figure 1) . The image memory 9 outputs a motion vector 
information (v=MV; motion vector) to the decoder. The DCT unit 
25 3 outputs the DCT coefficient (q) to the decoder. 






during the process of coding thej^.mag'esignal described above, 
especially during t^e^uantization process, thereby causing 
blocking artjJ€cts and ringing effects to the image which is 
reconstructed in- the decoder. The blocking artifacts imply 
irregularity between the blocks generated due to information 
loss resulting from the quantization of the low-frequency DCT 
coefficients, and the ringing effects results frona 
cmaftfTization errors of the high-frequency DCT ctre^#are^h=rTrf^ 

That is, in accordance with a coding technique using the 
DCT in a coding system of a static image or dynamic image, an 
image is divided into a plurality of blocks, and the DCT is 
performed on each block. On the other hand, when the DCT is 
carried out on the original image, its important information 
is mainly included in low- frequency elements, and becomes 
lesser in high-frequency elements. Furthermore, the low- 
frequency elements include a lot of information relating to 
adjacent blocks. The DCT does not consider the correlation 
between the blocks, and quantizes the low- frequency elements 
by blocks, thereby destroying continuity of the adjacent 
blocks. It is called the blocking artifacts. 

In addition, when the coefficients obtained by performing 
the DCT are quantized, as a quantization interval is 
increased, the elements to be coded are decreased, and thus 
the number of the bits to be processed is reduced. As a 
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result, the information of the high-frequency element included 
in the original image is reduced, thereby generating 
distortion of the reconstructed image. It is called the 
ringing effects. The ringing effects generated by increasing. 
5 the quantization interval are serious especially in a contour 
of an object in the reconstructed image. 

As techniques for removing the blocking artifacts and the 
ringing effects, employed are a low pass filtering method and 

1-3 a regularized image restoration method. 

*.3 

|>J 10 According to the low pass filtering method, a plurality 

of pixels around a predetermined pixel are selected, and an 
average value thereof is computed. Here, a filter tap or 
filter coefficients are set by experience. For example, 
referring to Figure 2, there is provided a block of N*N size. 
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,.Z 15 Reference numerals A to F depict pixels. Pixels C, D are 

\5 - 

*:3 adjacent to a boundary of the block. In order to reduce 

irregular variations between the pixels C, D, a k-tap (here, 

7-tap) filtering is performed, and a threshold value replacing 

a D pixel value is computed according to local statistics. 

2 0 There is an advantage in that a computation amount is reduced 
by utilizing a predetermined threshold value according to the 
comparison with the local statistics. However, an adaptive 
processing power in accordance with a quantization parameter 
is deficient, and thus a screen quality of the restored image 

25 is excessively smoothed according to the kind of the images 
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and compression ratio. 

The regularized image restoration method adaptively deals 
with the blocking artifacts in accordance with statistical 
properties of the image. That is, irregular information around 
the boundary of the block or in the block is all computed. 
However, the computed values form a matrix shape, and thus a 
real time processing is difficult due to the great computation 
amount. In addition, an average value obtained by a 
computation result of the irregular information is equally 
applied to the pixels, regardless of a degree of irregularity. 
Accordingly, when a block has a high degree of irregularity, 
it can be reduced. However, in case of a block having a low 
degree of irregularity, it may be increased. Thus, the system 
is not adaptive. Also, the information in the temporal section 
is not processed, and thus irregularity between the images 
cannot be adaptively processed. 



SUMMARY OF THE INVENTION 
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to provide a method^^'f f ^ : estoring a compressed image of an 
image pro^e-s-sing system and an apparatus therefor which can 
edue€ the blocking artifacts and ringing effects generated in 
^>r^§t o r e d i m3rge--s4g-aa-a^ — ^ 

It is another object of the present invention to provide 
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a method for restoring a compressed image of an image 
processing system and an apparatus therefor which consider a 
smoothing degree of an image and reliability for an original 
image by pixels having an identical property in image block 
5 units, during a decoding process. 

In order to achieve the above-described objects of the 
present invention, there is provided a method for restoring a 
compressed image of an image processing system including: a 
step of defining a smoothing functional having a degree of 
0 smoothing an image and reliability for an original image by 
pixels having an identical property in image block units; and 
a step of computing a restored image by performing a gradient 
operation on the smoothing functional in regard to the 
original image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will become better understood with 
reference to the accompanying drawings which. are given only by 
0 way of illustration and thus are not limitative of the present 
invention, wherein: 

Figure 1 is a block diagram illustrating an image encoder 
according to a reflated art; 

Figure 2 illustrates pixels in order to explain a low 
5 pass filtering method carried out in the image encoder of 
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Figure 1 ; 

i-s- a block diayiaui illusirira^Tig=a^^ppa-ra^ ; i3=^r^ = . 
restoring a comp^ processing system in 

accordance^wlth the present invention; 

ogure 4 illustrates a configuration of original pixels 
in A block of an original image in accordance with the present 
i^v^-n-fed^njL--^ 

Figure 5 illustrates directions of the irregular 
smoothing degree of the pixels in accordance with the present 
invention ; 

Figure 6 illustrates an image moved and compensated in 
regard to a temporal section in. accordance with the present 
invention; and 

feg-mre — =? — i -l-lustr-crt-es — a^B^wetegfc^e^ts^e — a-ppa^a-feu^Sx^ojc 
restorinqjs*[6^compressed image of the image processing system 
wrdsmce wicn the present — dnw-e nt ion . 



Figure 3 is a block diagram illustrating an apparatus for 
restoring a compressed image of an image processing system in 
accordance with the present invention. As shown therein, a 
decoder 210 receives INTRA/ INTER information (p=mtype) , 
transmission information (t) , quantization information 
(qz=Qp) , a discrete cosine transform (DCT) coefficient (q) and 
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motion vector information (v=MV; motion vector) from an 
encoder (as depicted in Figure 1) , and performs decoding. The 
encoder and the decoder 210 are connected by a communication 
channel or network. A post processing unit 220 receives image 
signals Y, U, V, a quantization variable (qz=Qp) , a macro 
block type (mtype) and a motion vector (v=MV) from the decoder 
210, and carries out an operation of restoring the compressed 
image in accordance with the present invention. 

According to the present invention, a smoothing 
functional is defined in regard to pixels having an identical 
property by blocks, a regularization parameter is computed 
based on the smoothing functional, and available values are 
applied to the regularization parameter, thereby obtaining an 
image to be restored. Thereafter, an iterative technique, a 
discrete cosine transform (DCT) , a projection and an inverse 
DCT are sequentially performed on the obtained image, thereby 
restoring a similar image to the original image. The whole 
processes will now be described in detail. 

Definition of Smoothing functional 

When an original image (f ) is compressed and transmitted, 
an image (g) reconstructed in the decoder 210 is represented 
by the following equation. 




g = f + n 



(1) 



Here, "g" and "f" indicate row vectors re -arranged in a 
^tack-order, namely a scanning order, and "n" indicates a 
5 quantization error. When it is presumed that a size of the 
image is MxM , the original image (f ) , the reconstructed image 
(g) and (n) are column vectors having a size of Mxl. 

An original pixel for the original image (f) is 
3 represented by f(i,j). Here, "i M and "j" indicate a position 

J 10 of the pixel in the image. 

J 

3 Figure 4 illustrates configuration of the original pixels 

" f(i,j) in the block of the original image (f) in order to 

j 

explain the present invention. Reference numerals in Figure 4 

3 

1 depict information of the respective pixels. 8x8 pixels are 

I 

ST 

S 15 shown in a single block. 

"a 

E? 

3 The 8x8 pixels in the block are classified into the 

pixels having an identical property. That is, the pixels are 
divided in accordance with their position, vertical direction, 
horizontal direction and smoothing variation in the temppral 
20 section. Accordingly, it is defined that a set of the pixels 
positioned at a boundary of the block in a vertical direction 
is C wo , a set of the pixels positioned inside the block in the 

VB 

vertical direction is C vg , a set of the pixels positioned at 
a boundary of a block in a horizontal direction is C HB , a set 
25 of the pixels positioned inside the block in the horizontal 
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direction is C HU(/ and a set of the pixels moved and compensated 
in the temporal section is C T . The sets C VB , C vw , C HB , C HU , C T are 
represented by the following expressions. 
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J 10 

j 

3 Here, the set C T is a set of the pixels having a macro 

* block type of "inter" or "not coded" in order to remove 

•a 

temporal redundancy information. 

3 

1 The smoothing functional M(f) for using the 

J 

S 15 regularizat ion restoration method from the above-defined sets 

3 C VB/ C vy , C HB , C Hg/ C T is defined as follows. 

M(f) = M VB (f) + M HB (f) + M w (f) + M HU (f) + M T (f) -- (3) 

20 Here, M VB (f) is a smoothing functional for the set C VB , 

M HB (f) is a smoothing functional for C HB , M w (f) is a smoothing 
functional for the set C VVJ/ M HU (f) is a smoothing functional 
for the set C HW , and M T (f) is a smoothing functional for the 
set C T . The smoothing fuctionals are respectively defined as 

25 follows. 

10 
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20 



\® X WQvwfW 2 = £(f(m,n)-f (m-l, n)) 2 ,m*0, 8, 16, . . 

' n-Cl m 



25 



M VB ( f ) = IIQ VB f« 2 + « VB ||g-f || 2 W1 

M HB< f > = HQHB f « 2 + <*HBU9- f « 2 W2 

M vw (f) = |Q W f|| 2 + Of^flg-f || 2 W3 

M HW (f) = ||Q HW f|| 2 + a HU ||g-f || 2 W4 



M T (f) = ||Q T f|| 2 + ajg-f' 2 
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Here, first terms in each expression indicate a smoothing 
degree .for the original pixel (reference pixel) -and adjacent 
3 pixel, and second terms indicate reliability for the original 

s? 

j 10 pixel and the restored pixel. " || ■ 11 " indicates the Euclidean 

J 

3 norm. Q VB , Q vu , Q HB , Q HW , Q T indicate high pass filters for 

S smoothing the pixels in the sets C VB , C vu , C HB , C HW , c T . 

J 

The first term at the right side is represented by the 
1 following expression. 



HQ^fH 2 = Vs(f (m,n)-f(m-l,n) ) 2 ,jii=0,8, 16, . . 

n=0m 



WQ^ff = z"x(f(in,n)-f(m r n-l)) 2 ,n=0,8,16,..'. -- (5) 

nm=0 ' 




M-l 

n-0m 



M-l 

WQ^ff = EE (f(m,n)-f(m,n-l) ) 2 ,n*0,8, 16, . . 

nm=0 
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||Q r fj| 2 = XX(f MC (m,n)-f (m,n) ) : 



L /fC 
n m 



The smoothing functionals represented by Expression (4) 
5 are quadratic equations, respectively. Thus, local minimizers 
of each smoothing functional become global minimizers. 

Figure 5 illustrates directions of the irregular 
smoothing degree of the pixels. There are a single pixel at 

!? the center and eight pixels therearound. There are also shown 

3 

s ' 10 horizontal and vertical arrows starting from the pixel at the 
J 

center. The arrows respectively depict the directions of the 
irregular smoothing degree in regard to the four adjacent 
pixels. That is to say, the irregular smoothing degree is 
considered in four directions in respect of a single pixel. 
15 Figure 6 illustrates an image moved and compensated in 

regard to the temporal section in accordance with the present 
invention. Arrows depict the correlation of a currently- 
restored image with a previously-restored image and a 
succeedingly reconstructed image, respectively. 
20 a? VB , a HB , c* vw , of HU , o; T included in the second terms of 

Expression (4) are regularizat ion parameters in regard to each 
set, indicate a ratio of the smoothing degree and reliability, 
and imply an error element. Wl, W2 , W3 , W4 , W5 indicate 
diagonal matrixes having a size of MxM in order to determine 
25 whether each set has an element, and have a value of "1" or 

12 
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"0" according to whether each pixel is included in a 
corresponding set. That is, if the respective pixels are 
included in the corresponding sets, the value of the diagonal 
elements is "1". If not, the value of the diagonal elements is 
" 0 " . 

Thereafter, the regularization parameters, a VB , o? HB , a vw , 
of HW , Qf T are approximated as follows. 

Approximation of Regularization Parameters 

Approximation of the regularization parameters is a major 
element determining performance of the smoothing functional. 
In order to reduce the computation amount, presumptions are 
made as follows. 

(1) A maximum value of the quantization error generated in 
the quantization process of the DCT region is Qp, and thus it 
is presumed that the quantization variables Qp are regular in 
each macro block. For this, the maximum quantization error of 
the DCT coefficients of each macro block is regularly set to 
be Qp. 

(2) It is also presumed that the DCT quantization errors have 
the Gaussain distribution property in the spatial section. 

Under the above presumptions, in case a set theoretic is 
applied, each regularization parameter is approximated as 



follows 



a VB 
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20 ^ "Here, Q 2 p (m,n) is a quantization variable of a macro block 

including a (m,n)th pixel of a two-dimensional image. 

In Expression (6) , denominator terms of the respective 
regularization parameters are a sum of the energy for the 
quantization noise of the elements included in each group. As 

25 described above, the values of the regularization parameters 
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may be easily computed by applying the set theoretic under the 
two presumptions. 

Computing Pixels to be Restored from Smoothing Functional 

5 

Only the original image needs to be computed. However, 
the smoothing functional includes a square term of the 
original image. Accordingly, in order to compute the original 
3 image, a gradient operation is carried out on the smoothing 

J 10 functional in regard to the original image. A result value 

j 

3 thereof is " 0 " , and represented by the following expression. 

a 

s? 
i 

•4 

V f M(f) = 2Q T VB Q VB + 2Q T HB Q HB + 2Q T yw Q vw + 2Q T HU Q HU + 2Q T T Q T 

Ji 

1 - 2Gf VB W T 1 W l (g-f ) - 2a HB W T 2 W 2 (g-f ) - 2a vu W T J W 3 (g-f ) -- (7) 

I 15 - 2a HW W\w 4 (g-f ) - 2a T (g-f) = 0 

..5 

J 

Here, a superscript "T" indicates a transposition . of the 
matrix . 

A restored image, similar to the original image (f) can be 
20 obtained by Expression (7) . However, operation of an inverse 
matrix must be performed, and thus the computation amount is 
increased. Thus, in accordance with the present invention, the 
restored image is computed by an iterative technique which 
will now be explained. 

25 
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Iterative Technique 

When Expression (7) is iterated k times, an iterative 
solution f k+1 is represented by the following expression. 

5 

f k+1 = f k + 3 [Ag - Bf k ] , 

B = (Q T VB Q VB + Q T HB Q HB + Q T vw Qvw + Q'hwQhw + Q\Q T > + A 

1 

J 10 In Expression (8), "3" is a relaxation parameter having 

3 a convergence property. Expression (8) can be represented by 

« the following expression by computing consecutive iterative 

i 

solutions . 



= 15 < f k+ l - f k> = (I - B) (f k - f,.,) --- (9) 
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Here, "I" is an identity matrix, and the matrix B has a 
positive definite property. Therefore, when the following 
condition is satisfied, the iterative solutions are converged. 

||I - B|| < 1 (10) 

Expression (10) can be summarized as follows. 



25 a/T\ 0 < /3 < 2 (11) 
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In Expression (11), "A(A) ,! depicts an eigen value of the 



compute the eigen value A (A) . However, the high pass filters 
have a certain shape determined according to the positions of 
the respective pixels, regardless of the image. Accordingly, 
before computing Expression (8), the eigen value A (A) can be 
replaced by a fixed value. The value may be computed by a 
power method which has been generally used in interpretation 
of numerical values. 

For example, a computation process of an eigen value of 
an iterative solution will now be explained. 



Here, M x k n is a vector of Mxl, and "K" is a positive- 
definite symmetric MxM matrix. The eigen value A' of the 
matrix K is approximated as follows. 



In the above expression, if "k" is to infinity, the eigen 
value A' is approximated to a real value. 

Thus, the iterative solution represented by Expression 
(8) is computed. The next thing to be considered is a time of 



matrix A. A considerable amount of computation is required to 



x k+1 - Kx k 
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finishing the iterative technique, in order to determine the 
number of iteration. Here, two standards are set as follows. 

Firstly, a predetermined threshold value is set before 
starting iteration, an image obtained after iteration, namely 
a partially-restored image is compared with the previously-set 
threshold value, and it is determined whether the iteration 
technique is continuously performed according to a comparison 
result . 

Secondly, the iteration technique is performed as many as 
a predetermined number, and then finished. 

According to the first standard, a predetermined 
threshold value is set in performing iteration, and thus a 
wanted value is obtained. However, although the iteration 
number is increased, it may happen that the predetermined 
threshold value is not reached. On the other hand, the second 
standard is performed by experience, but can reduce a 
computation amount. Therefore, the two standards may be 
selectively used according to the design specification. 

Figure 7 is a flowchart of the apparatus f.or restoring 
the compressed image of the image processing system in 
accordance with the present invention. As shown therein, in 
the step SI, the quantization variable Qp and the image 
signals Y, U, V are inputted, and the regularization parameter 
is approximated as described above. In the step S2, the 
gradient operation is performed on the smoothing functional in 
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regard to the original image. In the step S3, an iterative 

solution, namely a wanted restored image is obtained by the 

iteration technique. In this step, employed are the image 

signals Y, U, V and the motion vector MV which is moved and 

5 compensated. 

In the step S4 , the DCT is performed on the restored 

image corresponding to the iterative solution f k+1 obtained in 

the step S3. An (u,v)th DCT coefficient of the two-dimensional 

3 restored image is expressed as F k+1 (u,v) , and must exist in the 

J 10 following section in accordance with a property of the 
J 

3 quantization process. 

A 

G(u,v) - Qp <; F k+1 (u,v) <; G(u,v) + Qp (12) 

3 
i 

j 

» 15 Here, "Qp" is a maximum quantization error as explained 

3 

2 above, and "G(u,v) M is a two-dimensional DCT coefficient 

obtained by performing the DCT on the reconstructed image (g) . 
The DCT coefficients F k+1 (u,v) and G(u,v) are represented as 
follows. In Expression (13), "B" indicates a block DCT. 

20 

F k+1 (u,v) = (Bf k+1 ) (u,v) , and G(u,v) = (Bg)(u,v) (13) 

In the step S6, a section of the DCT coefficient of the 
restored image is set as in Expression (12) . Accordingly, in 
25 case the DCT coefficient F k+1 (u,v) of the restored image is not 
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in the predetermined section, it must be projected as follows. 
A projection process is carried out in the step S7, and 
represented by Expression (14) . 



P(F k+1 (u,v)) = G(u,v) - Qp, if F k+1 (u,v) < G(u,v) - Qp 
P(F k+1 (u / v)) = G(u,v) + Qp, if F k+1 (u,v) > G(u,v) - Qp -- (14) 
P(F k+1 (u,v)) = F k+1 (u,v), otherwise. 

Expression (14) will now be described. 

When F k+1 (u,v) is smaller than G(u,v) - Qp, the projected 
restored image P(F k+1 (u,v)) is mapped to G(u,v) - Qp . In case 
F k+1 (u,v) is greater than G(u,v) + Qp, the projected restored 
image P(F k+1 (u,v)) is mapped to G(u,v) + Qp . Otherwise, the 
projected restored image P(F k+1 (u,v)) is mapped as it is. 

In the step S8, the inverse DCT is performed on the 
mapped image P(F k+1 (u,v)) in the spatial section. The finally 
restored image is represented by Expression (14) . 

f k+1 = B T PBf k+1 (15) 

Here, "B" indicates the DCT, P indicates mapping, and 
"B Tm indicates the inverse DCT. 

The restored image is stored in a frame memory in the 
post processing unit 220 (Step S9) . The post processing unit 
220 performs motion compensation based on the motion vector MV 

20 



(Step S10) . The motion and compensation image is employed for 
generation of the regularization parameter for a succeeding 
image and the iteration technique. 

The post processing unit 220 outputs the restored motion 
5 and compensation image as a video signal to a display (not 
shown) (Step Sll) . 

As discussed earlier, the present invention can restrict 
a section of the restored image for the respective pixels by 

3 using the various regularization parameters. In addition, the 

3 

J 10 present invention prevents flickering which may occur in the 

^ dynamic image compression technique. 

zs 

Consequently, the present invention adaptively prevents 
the blocking artifacts and the ringing effects for the pixels 
1 having an identical property in image block units, and thus 

15 can be widely used for the products of the hybrid MC-DCT 
mechanism. 

As the present invention may be embodied in several forms 
without departing from the spirit or essential characteristics 
thereof, it should also be understood that the above-described . 
20 embodiment is not limited by any of the details of the 
foregoing description, unless otherwise specified, but rather 
should be construed broadly within its spirit and scope as 
defined in the appended claims, and therefore- all changes and 
modifications that fall within the meets and bounds of the 
25 claims, or equivalences of such meets and bounds are therefore 
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intended to be embraced by the appended claims. 



10 



15 



20 



22 




